Learning processes in neural networks.

We study the learning dynamics of neural networks from a general point of view. The environment from which the network learns is defined as a set of input stimuli. At discrete points in time, one of these stimuli is presented and an incremental learning step takes place. If the time between learning steps is drawn from a Poisson distribution, the dynamics of an ensemble of learning processes is described by a continuous-time master equation. The ensemble description allows us to study the asymptotic behavior of the plasticities for a large class of neural networks. For small learning parameters we derive an expression for the size of the fluctuations in an unchanging environment. We use the networks of Grossberg [19] and Oja [52] as simple examples to analyze and simulate the performance of neural networks. This chapter is an edited version of the first part of the paper " Learning processes in neural networks" by Tom Heskes and Bert Kappen, which has been published in Physical Review A, 44:2718-2726 [28]. Part of this work has been presented at the International Joint Conference on Neural Networks 1991 in Seattle and the International Conference on Artificial Neural Networks 1991 in Helsinki [32].